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Abstract 

We review the results of the Standard Model Higgs boson search at 
LEP. An emphasis is put on revealing the details behind the statistical 
procedure developed by the LEP Higgs working group. The procedure 
is explained using a toy model which allows the reader to estimate the 
significance of the experimental observation which led at the time to a 
scientific debate on whether LEP has observed a 115 GeV Higgs boson. 


1 Introduction 


On 3 November 2000 in a seminar at CERN the LEP Higgs working group pre¬ 
sented preliminary results of an analysis indicating a possible 2.9a observation 
of a 115 GeV Higgs boson [1]. Based on this analysis the four LEP collabora¬ 
tions requested the continuation of LEP to collect more data at ^/s = 208 GeV. 
However, the arguments presented by the LEP collaborations did not convince 
the LEP management and in retrospect, it turned out that the LEP accelerator 
turn-off date of 2 November 2000 ended its eleven years of forefront research. 

Figure 1, taken from the above mentioned presentation, shows the recon¬ 
structed mass distribution of the background and the signal (on top of the back¬ 
ground) with the data represented by the dots with error bars. It is shown for 
three possible selections of the data samples with increasing signal purity. None 
of these distributions shows a clear classical 3a excess of data over the expected 
background and some physicists claimed that this evidence was not convincing 
enough. However, the statistical arguments presented by the LEP Higgs working 
group were not based on these distributions, but rather on a sophisticated, though 
beautiful statistical analysis of the data. Two years after the event, when the last 
analysis of the LEP data indicated that the signihcance of a Higgs observation in 
the vicinity of 115 GeV went down to less than 2a [2], it becomes apparent that 
the LEP Standard Model (SM) Higgs heritage will in fact be a lower bound on 
the mass of the Higgs boson. However, the LEP Higgs working group has taught 
us powerful and instructive lessons of statistical methods for deriving limits and 
conhdence levels in the presence of mass dependent backgrounds from various 
channels and experiments. These lessons will remain with us long after the lower 
bound becomes outdated. 

In this note we are trying to repeat, in a pedagogical way, this LEP Higgs sta¬ 
tistical lesson. To achieve this we developed a toy Monte Garlo model containing 
a simulated background and signal similar to LEP conditions at y/s = 206 GeV. 

Due to mass resolution effects an event with a reconstructed mass rurec can 
originate from any Higgs mass in its vicinity. To appreciate the significance of 
a Higgs candidate with respect to some test mass mn the event weights are 
introduced in section 2. 

The weights of all candidate events are summed up in section 3 to give the 
likelihood of an experiment in order to quantify its signal-like nature. 

The probability density function of the likelihood is used in section 4 to esti¬ 
mate the probability of an experiment without a Higgs boson to fluctuate and give 
a more signal-like outcome than the observed one or the probability of an exper¬ 
iment in the presence of a Higgs signal to fluctuate and give a more background- 
like outcome than the observed one. Gonhdence levels are introduced for that 
purpose. Later on in that section, signal conhdence levels are dehned and used 
to estimate the exclusion sensitivity of an experiment. 

The current LEP SM Higgs search results are also given, allowing the reader 
to make his own appreciation and judgement. 


1 



2 Event Weights or “the Spaghetti Higgs Fac¬ 
tory” 

When does a Higgs candidate event become significant? What can be considered 
as a leading or gold plated Higgs candidate? Due to detector resolution and 
missing energy carried by undetected neutrinos which accompany the Higgs decay 
products, the reconstructed mass of a hypothetical Higgs candidate nirec, is not 
necessarily its physical mass, mn- One therefore defines a weight to quantify 
the significance of a Higgs candidate with rrirec with respect to a hypothetical 
Higgs with rriH- This weight is given by ln(l + (see next section) 

where s{mH,'mrec) is the expected number of signal events from an hypothetical 
Higgs boson with a test mass mn in the vicinity of nirec-^ This procedure is 
illustrated in a series of plots (Figure 2). In this example we assume there is an 
event candidate with a reconstructed mass, rrirec = HO GeV. We then calculate 
the event weight for hypothetical Higgs test masses in the range 100-120 GeV 
(Figs 2a-e). The resulting weights are shown in the last figure of this series (Fig. 
2e). When connecting all the weights with a continuous line, one gets what has 
become to be known as the “spaghetti plot”. Also shown is the spaghetti plot 
for a candidate event with rrirec = 113 GeV. As expected, the highest weight 
is achieved when the reconstructed mass and the hypothetical test Higgs mass 
coincide. The 17 candidates with the highest weight at a test mass, mn = 115 
GeV are listed in Table 1 and their corresponding spaghetti plots are shown in 
Figure 3 (taken from [2]). 

3 Understanding Likelihood Plots 

At the end of the day, LEP is one big experiment with one experimental result. 
An experimental result is in this sense a configuration of data events that agree 
to some level the expectation from either a pure background (6) hypothesis 
or signal plus background (s+b) hypothesis (at some Higgs mass). Here we 
illustrate how the likelihood ratio is used to rank an experimental result between 
either being 6-hke or s+ft-like. 

The first step in telling a 6-like from an s-|-6-like result is to construct a dis¬ 
criminator. Such a discriminator could be the reconstructed mass (obviously a 
peak at the reconstructed mass on top of the background will indicate a “signal 
observation”). It could also be a 2-D discriminator, where the other discriminat¬ 
ing variable is, for example, the b-tag content of an event. The Higgs, being the 
generator of particle masses, decays dominantly to b-quarks in this mass range. 
In our toy model, we use the reconstructed mass as a discriminator. 

^The histograms and consequently the weights are binned taking into account the experi¬ 
mental resolution and the statistics. 
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Expt 

Ecm 

Decay channel 

^^rec (GgV) 

ln(l 4- s/h) 
at 115 GeV 

1 

ALEPH 

206.6 

4-jet 

114.1 

1.76 

2 

ALEPH 

206.6 

4-jet 

114.4 

1.44 

3 

ALEPH 

206.4 

4-jet 

109.9 

0.59 

4 

L3 

206.4 

E-miss 

115.0 

0.53 

5 

ALEPH 

205.1 

Lept 

117.3 

0.49 

6 

ALEPH 

206.5 

Tans 

115.2 

0.45 

7 

OPAL 

206.4 

4-jet 

111.2 

0.43 

8 

ALEPH 

206.4 

4-jet 

114.4 

0.41 

9 

L3 

206.4 

4-jet 

108.3 

0.30 

10 

DELPHI 

206.6 

4-jet 

110.7 

0.28 

11 

ALEPH 

207.4 

4-jet 

102.8 

0.27 

12 

DELPHI 

206.6 

4-jet 

97.4 

0.23 

13 

OPAL 

201.5 

E-miss 

108.2 

0.22 

14 

L3 

206.4 

E-miss 

110.1 

0.21 

15 

ALEPH 

206.5 

4-jet 

114.2 

0.19 

16 

DELPHI 

206.6 

4-jet 

108.2 

0.19 

17 

L3 

206.6 

4-jet 

109.6 

0.18 


Table 1: Properties of the candidates with the highest weight at mn = 115 GeV. Table 
is taken from [2], 


Once the discriminator is defined, we divide it into bins, i = 1,2,. ,Nhins 

each containing TVj observed candidates. The likelihood ratio —2lnQ{mH) tells 
us how much the outcome of an experiment is signal-like [5]. It is given by 

exp (—(stot + &tot)) /-Sj -|- 

exp(-6TOT) ,=1 V / ’ 

( 1 ) 

-2 In Qimn) = 2 stot “ ^ E In j , (2) 

which is a weighted sum of all the observed events. 

Next we demonstrate how to construct the Probability Density Function 
(p.d.f.) of the likelihood for an ensemble of background and signal-l-background 
experiments. 

The histograms in Figure 4 show the expected background and 115 GeV Higgs 
signal (on top of the background) distributions of the discriminating variable. 


T L(^s ~\~ 5) 

Ppoisson{Data\b) L{b) 

which is easily simplihed to 
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here taken to be the reconstructed mass. We then show hve possible outcomes 
of experiments, all generated without the signal. The experiments are numbered 
1-5. For each experiment we use the observed events conhguration (W) and 
the hypothetical Higgs test mass uih to calculate the likelihood —2\nQ{mH) 
following equation 2. The resulting likelihoods are shown at the bottom right 
plot. Only experiment 4 has some excess of events at the high mass region which 
results in an s+6-hke likelihood. The other 4 experiments result in a 6-hke 
likelihood. The likelihood p.d.f. is the histogram generated when performing 
a large number of background only experiments. As one can see (from simple 
areas considerations), in this “typical” toy example , the probability for a 6-only 
experiments to give a s+6-like likelihood is about 15% (not so small...)^. 

The same procedure was repeated for signal+background experiments (see 
Figure 5). The bottom right plot shows the likelihood of the hve numbered s+b 
experiments. As one can see, experiments 3 and 4 gave a background-like likeli¬ 
hood. Repeating the s+b experiments a large number of times yielded another 
p.d.f. from which we see that in this specihc toy example, there is about a 20% 
probability for a 115 GeV signal to give a 6-hke event conhguration^. The p.d.f’s 
of the s+b and 6-only experiments give us an indication of the discriminating 
power of the likelihood. 

This is illustrated in Figure 6 where the p.d.f’s of 6-only and s+b experiments 
are shown for mn = 112,115 and 118 GeV. For the lighter Higgs, the production 
cross section is high allowing a good separation between signal and background. 
For the heavier Higgs the signal production cross section is very low resulting in 
a weak separation. The dependence of the likelihood on the hypothetical test 
Higgs mass is illustrated in Figure 7 where the median of the p.d.f. distributions 
is shown for 6-only and s+b experiments. As the Higgs test mass increases the 
signal and background separation power decreases resulting in a reduced discovery 
potential. This median can serve as a reference measure to the signal-like nature 
of the experimental result. To demonstrate this, we planted into our background 
Monte Garlo sample a 115 GeV Higgs boson signal. We then generated the p.d.f. 
of this fake signal likelihood. The resulting p.d.f. median is also shown in Figure 
7. One can see, as expected, that this likelihood reaches a broad minimum at a 
test mass in the vicinity of mn = 115 GeV. Naturally, this likelihood coincides 
with that of the s+b experiments for a test mass of 115 GeV. 

Now that we have learned how to read the likelihood plots let us examine 
the LEP experimental outputs [2]. The likelihood outcome of the combined four 
LEP experiments is shown in Figure 8. This result can be sliced into the different 
experiments (Figure 9) or the different search channels (Figure 10) [2]. Shown are 
the 6-only median expectations with its 1 and 2(7 bands, the s-|-b expectation and 

^Later, in this note, we will see that this probability is 1 — CLb, where CLt, is the background 
confidence. 

^Later in this note, we will see that this probability is the signal-l-background confidence, 

CL s+b. 
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the observed likelihood. The broad minimum of the combined LEP likelihood 
from niH ~ 115 — 118 GeV which crosses the expectation for s+b around mn ~ 
116 GeV can be interpreted as a preference for a Standard Model Higgs boson 
at this mass range, however, at less than the 2a level. When the LEP Higgs 
working group presented these results for the hrst time the signihcance was 2.9a 
[1], and this relatively high significance generated a storm which unfortunately 
turned out to be in a tea cup... 

The ALEPH observed likelihood has a 3a signal-like behavior around rriH ~ 
114 GeV, which led the collaboration to claim a possible observation of a SM 
Higgs boson [3]. This behavior originated mainly from the 4-jet channel and its 
signihcance is reduced when all experiments are combined. No other experiment 
or channel indicated a signal-like behavior. 


4 Confidence Levels or How Probable is a Re¬ 
sult? 

Figure 11 shows the likelihood p.d.f. of our toy model with a test mass of mn ~ 
115 GeV. Assuming an hypothetical observed likelihood of —2lnQ = —3 the 
probability for a 6-only experiment to give a more s-|-&-like likelihood than the 
observed one is given by the area marked as l — CLf,, where CLh is the background 
conhdence level. It is easy to see that the expectation value of this probability 
is 50%, i.e. (1 — CLb) = 0.5 irrespective of the test Higgs mass. One might 
also say that CL), measures the compatibility with the background hypothesis. 
The combined LEP p.d.f. is given in Figure 12. For a test mass of mn = 116 
GeV, the observed likelihood is such that 1 — CL), = 0.099 [2], i.e., in 9.9% of 
background-only experiments, we expect to observe a result at least as signal- 
like as we observe. In the bottom plots of this Figure, the p.d.f’s are shown for 
Higgs test masses of 110 and 120 GeV where the observed probability is clearly 
consistent with the background hypothesis. These probabilities can be translated 
into a signihcance. A Gaussian approximation is used [4]. Table 2 shows the 


l-CL), 

0.32 

0.046 


6.3 X 10-^ 

QoQlJig 


la 

2a 

3a 

4a 

5a 


Table 2: Significances 


correspondence between 1 — CL), and the resulting signihcance. Therefore, the 
combined LEP observed likelihood corresponds to a signihcance below 2a. 

Figure 13 (top) shows the probability 1 — CL), as a function of the test mass 
mu [2]. Shown are the median probability expected for 6-only (dashed) and s+b 
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(dash-dotted) experiments, and the observed probability (solid red line). One 
can see that even though the observed probability is compatible with a ~116 
GeV Higgs boson, the sensitivity in this vicinity is less than 2a, and LEP did not 
really have the sensitivity to observe a Higgs boson heavier than 115 GeV with 
more than 3a signihcance (this can be seen by the intersection of the dash-dotted 
line with the horizontal 3a line). In fact. The LEP sensitivity to observe a SM 
Higgs boson at greater than 3a signihcance extends up to about 115 GeV and up 
to about 116 GeV for observation at greater than 2a. 

The bottom plots of Figure 13 show the probability 1 — CLb where the ALEPH 
excess of candidate events in the 4-jet channel manifests itself as a 3a deviation 
for a 116 GeV Higgs boson for ALEPH alone, and a 2a deviation in the combined 
LEP 4-jet channel. Note that ALEPH stand-alone sensitivity to observe a Higgs 
boson at the 3a level extends up to uih ~ 113 GeV, which is of course less than 
the combined LEP sensitivity. The ALEPH excess should therefore be interpreted 
as a huctuation in that context. 

In case there are no clear indications for discovery, one would like to interpret 
the search results in terms of exclusion. The probability CLg+b, shown as the 
blue area in the bottom plot of Figure 11, measures the compatibility of the 
experiment with the s+b hypothesis. There is no way to directly measure the 
signal Gonhdence Level, CLg because of the presence of signihcant background. A 
bigger CLs+b means the experimental result is more s-|-6-like, but not necessarily 
more s-like due to the relative fluctuations of the background. Therefore, if 
CLs+b is small, say, less than 5%, one can exclude the s+b hypothesis at more 
than 95% Gonhdence Level, but that does not mean that the signal hypothesis is 
excluded at that level. An example is given in Table 3 [2] where the background 



1-CLb 

CLg+b 

LEP 

0.099 

0.369 

ALEPH 

2.41 X 10“^ 

0.956 

DELPHI 

0.874 

0.033 

L3 

0.348 

0.408 

OPAL 

0.543 

0.208 

Four-jet 

5.70 X 10-^ 

0.676 

All but four-jet 

0.368 

0.217 


Table 3: The background conhdence level 1 — CLb and the signal-|-background 
conhdence level CLg+b at mu = 116 GeV, for all LEP data combined and for 
various subsets of the data. The numbers for the four-jet and all-but-four-jet 
hnal states are obtained with the data of the four experiments combined. Table 
taken from [2]. 


conhdence level 1 — CLb and the signal-|-background conhdence level CLg+b at 
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mn = 116 GeV, for all LEP data combined and for various subsets of the data 
are shown . Note that the DELPHI CLg+b probability is 3.3%. That does not 
mean that DELPHI could exclude a 116 GeV Higgs boson signal hypothesis, but 
rather that DELPHI can exclude a 116 GeV s+b hypothesis. However, there is a 
1 — CLi, = 87.4% probability for the DELPHI background to fluctuate and give 
a signal-like observation. It was to take this probability into account that one 
apriori defined the signal Gonhdence level to be CLg = [5]. This way the 

signal hypothesis for nih = 116 GeV is excluded by dehnition at only the 73.8% 
Gonhdence Level, CL = 1 — CLg = 1 — 0.033/0.126 = 0.738. This procedure 
for constructing the signal conhdence level is obviously conservative since the 
coverage probability is in general greater than the CL. 

Of the three plots shown in Eigure 12 it is clear now that a 110 GeV Higgs 
boson, where the CLg+b probability is too small to be seen, can be excluded by 
LEP. This is also seen in Table 3 where none of the experiments alone or the 
combined LEP can exclude a 116 GeV Higgs boson. 

The exclusion power of LEP is illustrated in Figure 14 [2] where the median ex¬ 
pected signal conhdence level, {CL\, as a function of the Higgs test mass together 
with its 1 and 2(j bands is shown. Also shown is the observed CLg conhdence 
level. The intersection of the horizontal line at CLs = 5% with the observed 
and median expected curves give the 95% observed and expected exclusion CL. 
Thus LEP excluded, at 95% CL., a SM Higgs boson with a mass below 114.4 
GeV, while it had the sensitivity to exclude a 115.3 GeV Higgs boson. The small 
excess of data candidate events around 115-116 GeV is responsible for the slight 
reduction of the observed Higgs lower mass limit with respect to its potential 
exclusion sensitivity. 

5 Conclusion 

LEP Standard Model Higgs search results were reviewed with the emphasis on a 
pedagogical explanation of the statistical procedure based on a toy model con¬ 
structed specihcally for this purpose. 
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Figure 1: Distributions of the reconstructed Higgs mass, rrirec-, obtained from 
three special, non-biasing, selections with increasing signal purity. In the 
loose/medium/tight selections the cuts are adjusted in such a way as to ob¬ 
tain, for a Higgs boson of 115 GeV mass, approximately 0.5/1/2 times as many 
expected signal as background events in the region rUrec > 109 GeV[l]. 
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Figure 2: The ’’spaghetti” curves represent the evolution of the event weight ln(l + s/6) 
with test mass niH- The first five plots show the histograms of the signal (red) on top 
of the background (yellow) for test masses from 100-120 GeV. The arrow represent the 
location of the candidate reconstructed mass at mu = 110 GeV. The bottom right plot 
shows the resulting spaghetti plot for a candidate with nirec = 110 and 113 GeV. 
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Figure 3: Evolution of the event weight ln(l + s/b) with test-mass mn for the events 
with the largest weight at mu = 115 GeV. The labels correspond to the candidate 
numbers in the first column of Table 1. The sudden increase in the weight of the 
OPAL missing-energy candidate labeled “13” at mn = 107 GeV is due to the 
switching from the low-mass to high-mass optimization of the search at that mass. 
A similar increase is observed in the case of the L3 four-jet candidate labeled “17” 
which is due to a test-mass dependent attribution of the jet-pairs to the Z and 
Higgs bosons.The Figure is taken from [2]. 
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Figure 4: The first five plots show the histograms of the 115 GeV Higgs signal (red) 
on top of the background (yellow) for a Higgs test mass of 115 GeV. The dots with 
error bars represent a simulated data of random background only experiments. Each 
experiment yields a likelihood represented by the arrows in the bottom right plot. By 
performing an infinite number of background only experiments one gets the probability 
density function shown in the bottom right plot. 
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Figure 5: The first five plots show the histograms of the 115 GeV Higgs signal (red) 
on top of the background (yellow) for a Higgs test mass of 115 GeV. The dots with 
error bars represent simulated data of random signal+background experiments. Each 
experiment yields a likelihood represented by the arrows in the bottom right plot. 
By performing a large number of signal+background only experiments one gets the 
probability density function(p.d.f.) shown in the bottom right plot. Also shown is the 
background only p.d.f. 
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Figure 6: The separation between the Signal and the Background for various 
Higgs masses is shown by their likelihood p.d.f’s. 
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Figure 7: Median expected likelihood of background-only (blue), signal(at the 
test mass mH)-|-background (red) and a fake 115 GeV Higgs signal (green) as a 
function of the test Higgs mass mn- 
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m^( GeV/c^) 

Figure 8: Observed and expected behavior of the likelihood —21nQ as a 
function of the test-mass niH for combined LEP experiments. The solid/red 
line represents the observation; the dashed/dash-dotted lines show the median 
background/signal-|-background expectations. The dark/green and light/yellow 
shaded bands represent the 1 and 2 a probability bands about the median back¬ 
ground expectation [2]. 
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Figure 9: Observed and expected behavior of the likelihood —2lnQ as a function of 
the test-mass mn for the various experiments. The solid/red line represents the obser¬ 
vation; the dashed/dash-dotted lines show the median backgronnd/signal-|-background 
expectations. The dark/green and light/yellow shaded bands represent the 1 and 2 a 
probability bands about the median background expectation [2]. 
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Figure 10: Observed and expected behavior of the likelihood —21nQ as a func¬ 
tion of the test-mass niH for the various Higgs search channels. The solid/red 
line represents the observation; the dashed/dash-dotted lines show the median back¬ 
ground/signal-^background expectations. The dark/green and light/yellow shaded 
bands represent the 1 and 2 a probability bands about the median background ex¬ 
pectation [2]. 
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Figure 11: An example of probability density functions (p.d.f’s) for background 
only (solid blue) and signal+background (solid red) experiments. The red shaded 
area, 1 — CLh, measure the compatibility with the background hypothesis while 
the blue shaded area, the compatibility with the signal+background hy¬ 

pothesis. Detailed explanations are given in the text. 
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Figure 12: Probability density functions corresponding to fixed test-masses, for the 
background and signal-|-background hypotheses. The observed likelihood ratio —2\nQ 
is indicated by the vertical line. The light/red shaded areas, 1 — measure the 

compatibility with the background hypothesis and the dark/blue shaded areas, CLg+b-, 
the compatibility with the signal+background hypothesis. Upper part: test-mass 
mu = 116 GeV; lower part: mu = 110 (left) and 120 GeV (right) [2]. 
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Figure 13: Probability 1 — CLb as a function of the test-mass mn- Solid/red 
line: observation; dashed/dash-dotted lines: expected probability for the back¬ 
ground/signal-|-background hypotheses. The horizontal solid lines indicate the levels 
for 2a, 3a and 4(7 deviations from the background hypothesis. The top plot is the com¬ 
bined LEP result while the bottom plots show the ALEPH and the LEP 4-jet channel 
results [2]. 
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Figure 14: Confidence level CLg for the signal+background hypothesis. Solid/red 
line: observation; dashed line: median background expectation. The dark/green and 
light/yellow shaded bands around the median expected line correspond to the 1 and 2 a 
probability bands computed with a large number of simulated background experiments. 
The intersection of the horizontal line at CLg = 0.05 with the observed curve defines the 
95% confidence level lower bound for the mass of the Standard Model Higgs boson[2]. 
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